Compression offloading to RAID array storage enclosure

ABSTRACT

A storage system comprises a plurality of enclosures and a storage controller. Each enclosure comprises at least one processing device and a plurality of drives configured in accordance with a redundant array of independent disks (RAID) arrangement. The storage controller obtains data pages associated with an input-output request, provides the data pages to a processing device of a given enclosure, and issues a command to the processing device to perform at least one operation based at least in part on the data pages. The processing device of the given enclosure receives the data pages from the storage controller, generates compressed data pages based at least in part on the received data pages, stores one or more of the compressed data pages on the plurality of drives according to the RAID arrangement and returns information associated with the storage of the compressed data pages to the storage controller.

FIELD

The field relates generally to information processing systems, and moreparticularly to storage in information processing systems.

BACKGROUND

In many information processing systems, storage systems are keyelements. Storage systems, such as block based storage systems, aredesigned to store and retrieve large amounts of data. To store a blockof data, a host device typically provides a data block address and datablock content to a storage system. The host device also provides thedata block address to the storage system to retrieve the data blockcontent stored in the storage system at a physical address. Some storagesolutions rely on address-based mapping of data, as well asaddress-based functionality of a storage system's internal algorithms.For example, computing applications typically rely on address-basedmapping and identification of data that is stored and retrieved. Anothersolution, in which data is mapped internally within a storage system andmanaged based on its content instead of its address, can provide varioussubstantial advantages. For example, such a content-based storagesolution improves storage capacity efficiency since any duplicate datablocks will only occupy the actual capacity of a single instance of thatdata block. As another example, the content-based storage solution canimprove performance since duplicate block writes do not need to beexecuted internally in the storage system. Content-based storagesolutions, however, face various challenges.

SUMMARY

In some embodiments, a storage system comprises a plurality ofenclosures and a storage controller. Each enclosure comprises at leastone processing device coupled to memory and a plurality of drivesconfigured in accordance with a redundant array of independent disks(RAID) arrangement. The storage controller is configured to obtain datapages associated with at least one input-output request and to providethe obtained data pages to the at least one processing device of a givenenclosure of the plurality of enclosures. The storage controller isfurther configured to issue a command to the at least one processingdevice of the given enclosure to perform at least one operation based atleast in part on the obtained data pages. The at least one processingdevice of the given enclosure is configured to receive the obtained datapages from the storage controller and responsive to receiving thecommand from the storage controller, to generate compressed data pagesbased at least in part on the received data pages. The at least oneprocessing device of the given enclosure is further configured to storeone or more of the compressed data pages on the plurality of drivesaccording to the RAID arrangement and to return information associatedwith the storage of the one or more of the compressed data pages to thestorage controller. The storage controller is further configured toutilize the information to access the one or more of the compressed datapages stored on the plurality of drives.

These and other illustrative embodiments include, without limitation,apparatus, systems, methods and processor-readable storage media.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram of one example of an information processingsystem within which one or more illustrative embodiments areimplemented.

FIG. 2 is a block diagram of another example of an informationprocessing system within which one or more illustrative embodiments areimplemented.

FIG. 3 is a block diagram of one example of a storage system withinwhich one or more illustrative embodiments are implemented.

FIG. 4 is a flow diagram illustrating part of a methodology for writeflow offloading in an illustrative embodiment.

FIG. 5 is a flow diagram illustrating part of a methodology forcompression offloading in an illustrative embodiment.

FIG. 6 shows a content addressable storage system having a distributedstorage controller configured with functionality for performing writeflow offloading and compression offloading in an illustrativeembodiment.

FIGS. 7 and 8 show examples of processing platforms that may be utilizedto implement at least a portion of an information processing system inillustrative embodiments.

DETAILED DESCRIPTION

Illustrative embodiments will be described herein with reference toexemplary information processing systems and associated computers,servers, storage devices and other processing devices. It is to beappreciated, however, that these and other embodiments are notrestricted to the particular illustrative system and deviceconfigurations shown. Accordingly, the term “information processingsystem” as used herein is intended to be broadly construed, so as toencompass, for example, processing systems comprising cloud computingand storage systems, as well as other types of processing systemscomprising various combinations of physical and virtual processingresources. An information processing system may therefore comprise, forexample, at least one data center or other cloud-based system thatincludes one or more clouds hosting multiple tenants that share cloudresources. Numerous different types of enterprise computing and storagesystems are also encompassed by the term “information processing system”as that term is broadly used herein.

FIG. 1 shows an information processing system 100 configured inaccordance with an illustrative embodiment. The information processingsystem 100 comprises a host device 102, which may comprise one of aplurality of host devices of a computer system. The host device 102communicates over a network 104 with a storage system 105. The hostdevice 102 and storage system 105 may be part of an enterprise computingand storage system, a cloud-based system or another type of system.

The host device 102 and storage system 105 illustratively compriserespective processing devices of one or more processing platforms. Forexample, the host device 102 and the storage system 105 can eachcomprise one or more processing devices each having a processor and amemory, possibly implementing virtual machines and/or containers,although numerous other configurations are possible.

The host device 102 and the storage system 105 can additionally oralternatively be part of cloud infrastructure such as an Amazon WebServices (AWS) system. Other examples of cloud-based systems that can beused to provide one or more of host device 102 and storage system 105include Google Cloud Platform (GCP) and Microsoft Azure.

The host device 102 is configured to write data to and read data fromthe storage system 105. The host device 102 and the storage system 105may be implemented on a common processing platform, or on separateprocessing platforms. A wide variety of other types of host devices canbe used in other embodiments.

The host device 102 in some embodiments illustratively provides computeservices such as execution of one or more applications on behalf of eachof one or more users associated with the host device 102.

The term “user” herein is intended to be broadly construed so as toencompass numerous arrangements of human, hardware, software or firmwareentities, as well as combinations of such entities. Compute and/orstorage services may be provided for users under a Platform-as-a-Service(PaaS) model, although it is to be appreciated that numerous other cloudinfrastructure arrangements could be used. Also, illustrativeembodiments can be implemented outside of the cloud infrastructurecontext, as in the case of a stand-alone computing and storage systemimplemented within a given enterprise.

The network 104 is assumed to comprise a portion of a global computernetwork such as the Internet, although other types of networks can bepart of the network 104, including a wide area network (WAN), a localarea network (LAN), a satellite network, a telephone or cable network, acellular network, a wireless network such as a WiFi or WiMAX network, orvarious portions or combinations of these and other types of networks.The network 104 in some embodiments therefore comprises combinations ofmultiple different types of networks each comprising processing devicesconfigured to communicate using Internet Protocol (IP) or othercommunication protocols.

As a more particular example, some embodiments may utilize one or morehigh-speed local networks in which associated processing devicescommunicate with one another utilizing Peripheral Component Interconnectexpress (PCIe) cards of those devices, and networking protocols such asInfiniBand, Gigabit Ethernet, Fibre Channel, or Non-Volatile Memoryexpress Over Fabrics (NVMeOF). Numerous alternative networkingarrangements are possible in a given embodiment, as will be appreciatedby those skilled in the art.

The storage system 105 is accessible to the host device 102 over thenetwork 104. The storage system 105 comprises a plurality of storageenclosures 106, an associated storage controller 108, and an associatedcache 109.

The storage enclosures 106 illustratively comprises storage devices,such as, e.g., solid state drives (SSDs). Such SSDs are implementedusing non-volatile memory (NVM) devices such as flash memory. Othertypes of NVM devices that can be used to implement at least a portion ofthe storage devices 106 include non-volatile random access memory(NVRAM), phase-change RAM (PC-RAM) and magnetic RAM (MRAM). These andvarious combinations of multiple different types of NVM devices may alsobe used.

However, it is to be appreciated that other types of storage devices canbe used in other embodiments. For example, a given storage system as theterm is broadly used herein can include a combination of different typesof storage devices, as in the case of a multi-tier storage systemcomprising a flash-based fast tier and a disk-based capacity tier. Insuch an embodiment, each of the fast tier and the capacity tier of themulti-tier storage system comprises a plurality of storage devices withdifferent types of storage devices being used in different ones of thestorage tiers. For example, the fast tier may comprise flash driveswhile the capacity tier comprises hard disk drives. The particularstorage devices used in a given storage tier may be varied in otherembodiments, and multiple distinct storage device types may be usedwithin a single storage tier. The term “storage device” as used hereinis intended to be broadly construed, so as to encompass, for example,flash drives, solid state drives, hard disk drives, hybrid drives orother types of storage devices.

In some embodiments, the storage system 105 illustratively comprises ascale-out all-flash content addressable storage array such as anXtremIO™ storage array from Dell EMC of Hopkinton, Mass. Other types ofstorage arrays, including by way of example VNX® and Symmetrix VMAX®storage arrays also from Dell EMC, can be used to implement storagesystem 105 in other embodiments.

The term “storage system” as used herein is therefore intended to bebroadly construed, and should not be viewed as being limited to contentaddressable storage systems or flash-based storage systems. A givenstorage system as the term is broadly used herein can comprise, forexample, network-attached storage (NAS), storage area networks (SANs),direct-attached storage (DAS) and distributed DAS, as well ascombinations of these and other storage types, includingsoftware-defined storage.

Other particular types of storage products that can be used to implementstorage system 105 in illustrative embodiments include all-flash andhybrid flash storage arrays such as Unity™ software-defined storageproducts such as ScaleIO™ and ViPR®, cloud storage products such asElastic Cloud Storage (ECS), object-based storage products such asAtmos®, and scale-out NAS clusters comprising Isilon® platform nodes andassociated accelerators, all from Dell EMC. Combinations of multipleones of these and other storage products can also be used inimplementing a given storage system in an illustrative embodiment.

In the FIG. 1 embodiment, the storage enclosures 106 implement one ormore RAID arrays of storage devices, denoted as RAID arrays 110. TheRAID arrays 110 may collectively form a single RAID array 110, or mayrepresent distinct RAID arrays. The RAID arrays 110 are assumed to storedata in stripes across a plurality of SSDs or other storage devicesprovided by the storage enclosures 106. The RAID array 110 is an exampleof what is more generally referred to herein as data striping across aplurality of storage devices in a storage system.

In the FIG. 1 embodiment, the storage enclosures 106 also implement oneor more processing devices 112. The processing devices 112 may comprisea microprocessor, a microcontroller, an application-specific integratedcircuit (ASIC), a field-programmable gate array (FPGA), graphicsprocessing unit (GPU) or other type of processing circuitry, as well asportions or combinations of such circuitry elements.

The cache 109 of storage system 105 in the FIG. 1 embodiment includescache entries which store incoming input-output (IO) request data forlater destaging to storage devices of the storage enclosures 106. Cache109 may illustratively comprise volatile memory such as, e.g., randomaccess memory (RAM), dynamic random-access memory (DRAM), staticrandom-access memory (SRAM), or any other kind of volatile memory. Insome embodiments, cache 109 may additionally or alternatively compriseany non-volatile memory as described above with respect to storagedevices 106. In some embodiments, cache 109 may support a variety ofoperations or functions of storage system 105 including, for example,write cache, read cache, temporary metadata storage, or other similaroperations. While illustrated as a separate component of storage system105, in some embodiments, cache 109 may be included as a component ofstorage controller 108 or a component of each storage enclosure 106. Insome embodiments, the caches 109 of storage system 105 may operatetogether as a single cache 109 where the components of a given storagesystem 105 may access any portion of the cache 109 including thoseportions included as components of other portions of storage system 105.

In an illustrative embodiment, as illustrated in FIG. 1, the host device102 includes write flow offload logic 114 which provides logic andfunctionality for offloading some or all of the RAID processing of datapages for write requests from the storage controller 108 to theprocessing devices 112 of the storage enclosures 106. By offloading RAIDprocessing, the availability of the storage controller 108 for servicingadditional input-output operations is increased. The write flow offloadlogic 114 will be described in more detail below.

In another illustrative embodiment, as also illustrated in FIG. 1, thehost device 102 includes compression offload logic 116 which provideslogic and functionality for offloading some or all of the compressionprocessing of data pages from the storage controller 108 to theprocessing devices 112 of the storage enclosures 106. By offloadingcompression processing, the availability of storage controller 108 forservicing additional input-output operations is increased. Thecompression offload logic 116 will be described in more detail below.

While described as separate embodiments, in an illustrative embodiment,the write flow offload logic 114 and compression offload logic 116 mayalternatively be implemented together by the host device 102.

The host device 102 should also be understood to include additionalmodules and other components typically found in conventionalimplementations of computers, servers or other host devices, althoughsuch additional modules and other components are omitted from the figurefor clarity and simplicity of illustration.

The host device 102 and storage system 105 in the FIG. 1 embodiments areassumed to be implemented using at least one processing platform eachcomprising one or more processing devices each having a processorcoupled to a memory. Such processing devices can illustratively includeparticular arrangements of compute, storage and network resources.

The host device 102 and the storage system 105 may be implemented onrespective distinct processing platforms, although numerous otherarrangements are possible. For example, in some embodiments at leastportions of the host device 102 and at least portions of the storagesystem 105 are implemented on the same processing platform. The storagesystem 105 can therefore be implemented at least in part within at leastone processing platform that implements at least a portion of the hostdevice 102.

The term “processing platform” as used herein is intended to be broadlyconstrued so as to encompass, by way of illustration and withoutlimitation, multiple sets of processing devices and associated storagesystems that are configured to communicate over one or more networks.For example, distributed implementations of the system 100 are possible,in which certain components of the system reside in one data center in afirst geographic location while other components of the system reside inone or more other data centers in one or more other geographic locationsthat are potentially remote from the first geographic location. Thus, itis possible in some implementations of the system 100 for the hostdevice 102 and storage system 105 to reside in different data centers.Numerous other distributed implementations of one or both of the hostdevice 102 and the storage system 105 are possible. Accordingly, thestorage system 105 can also be implemented in a distributed manneracross multiple data centers.

Additional examples of processing platforms utilized to implement hostdevices and/or storage systems in illustrative embodiments will bedescribed in more detail below in conjunction with FIGS. 6-8.

It is to be appreciated that these and other features of illustrativeembodiments are presented by way of example only, and should not beconstrued as limiting in any way.

Accordingly, different numbers, types and arrangements of systemcomponents such as host device 102, network 104, storage system 105,storage enclosures 106, storage controllers 108, cache 109, RAID arrays110 and processing devices 112 can be used in other embodiments.

It should be understood that the particular sets of modules and othercomponents implemented in the system 100 as illustrated in FIG. 1 arepresented by way of example only. In other embodiments, only subsets ofthese components, or additional or alternative sets of components, maybe used, and such components may exhibit alternative functionality andconfigurations. Additional examples of systems implementingfunctionality for write flow and compression offloading using theprocessing devices of the storage enclosures will be described below.

FIG. 2 shows an information processing system 200 configured inaccordance with other illustrative embodiments. The informationprocessing system 200 comprises a computer system 201 that includes hostdevices 202-1, 202-2, . . . 202-N collectively referred to as hostdevices 202. The host devices 202 communicate over a network 204 with astorage system 205. The computer system 201 is assumed to comprise anenterprise computer system, cloud-based computer system or otherarrangement of multiple compute nodes associated with respective users.The host devices 202 of the computer system 201 in some embodimentsillustratively provide compute services such as execution of one or moreapplications on behalf of each of one or more users associated withrespective ones of the host devices 202.

Similar to the storage system 105 of system 100, the storage system 205comprises storage enclosures 206, a storage controller 208, a cache 209,a RAID array 210, and processing devices 212. However, in theseembodiments, the functionality for write flow offloading and compressionoffloading is implemented in the storage system 205, rather than in oneof the host devices 202. Accordingly, the storage controller 208 inthese embodiments comprises one or both of write flow offload logic 214and compression offload logic 216, which are configured to operate insubstantially the same manner as that described above for write flowoffload logic 114 and compression offload logic 116 of the host device102 in the system 100. In some embodiments, functionality for write flowand compression offloading can be implemented partially in a host deviceand partially in the storage system. Accordingly, illustrativeembodiments are not limited to arrangements in which all suchfunctionality is implemented in a host device or a storage system, andtherefore encompass various hybrid arrangements in which thefunctionality is distributed over one or more host devices and one ormore storage systems, each comprising one or more processing devices.

In some embodiments, the processing devices 212 of the storageenclosures 206 may implement some or all of the functionality of writeflow offload logic 214, compression offload logic 216, or both. In someembodiments, the functionality of write flow offload logic 214 andcompression offload logic 216 may be implemented in part by theprocessing devices 212 and in part by the storage controller 208, hostdevices 202 or both.

Illustrative write flow and compression offloading operations will nowbe described in further detail in the context of the informationprocessing systems 100 and 200. However, it is to be understood thatwrite flow and compression offloading are more generally applicable toother types of information processing systems. At least some of thewrite flow and compression offloading steps are illustratively performedunder the control of the write flow offload logic 114 and compressionoffload logic 116 in host device 102 of system 100, in write flowoffload logic 212 and compression offload logic 214 in storagecontroller 208, processing devices 212, or both, of system 200.

Data striping in some embodiments is implemented utilizing RAID, usingRAID arrays 110 on storage system 105 or RAID arrays 210 on storagesystem 205. In such embodiments, the number of data disks in the RAIDstorage system may comprise a prime number k, and a column of the RAIDstorage system comprises k−1 blocks. The storage devices of the RAIDstorage system may be SSDs. In some embodiments, the RAID storage systemmay implement RAID 6 with the number of data disks being k and thenumber of parity disks being n, where n is greater than one (e.g., wheren=2). In some embodiments, the stripe column size is selected as amultiple of a designated block size. The multiple may be a prime numberP minus 1. The prime number P may be the same as or different than theprime numbers selected for different ones of the stripes.

In some cases, the prime number selected for a particular stripe may begreater than a number of the plurality of storage devices in the storagesystem that store data blocks for that stripe. To handle suchsituations, the parity blocks for the stripe may be computed by assumingor setting a set of virtual storage devices with pages storingdesignated predetermined values (e.g., zero pages). The particularnumber of virtual storage devices in the set may be equal to thedifference between the prime number selected for that stripe and thenumber of storage devices in the storage system which store data blocksfor that stripe.

The term RAID, as used herein, is an umbrella term for computer datastorage schemes that can divide and replicate data among multiplephysical disk drives. The terms disks and drives will be usedinterchangeably henceforth. The physical disks are said to be in a RAIDarray, which is accessed by an operating system as one single disk. Thedifferent schemes or architectures are named by the word RAID followedby a number (e.g., RAID 0, RAID 1, etc.). Each scheme provides adifferent balance between the goals of increasing data reliability andincreasing input/output performance. While in some embodiments, thestorage system is described herein with reference to a RAID array havinga RAID 6 scheme, any other RAID scheme may be used in the disclosedembodiments.

The RAID 6 scheme was developed to provide functionality for recoveringfrom multiple disk failure (e.g., similar to RAID 1.3) with highutilization rates (e.g., comparable to RAID 4 and 5) that avoids systembottlenecks. RAID 6 uses an N+2 parity scheme, which allows failure oftwo disks, where N is the number of disks in the array. RAID 6 definesblock-level striping with double distributed parity and provides faulttolerance of two drive failures, so that the array continues to operatewith up to two failed drives, irrespective of which two drives fail.

There are various implementations of RAID 6, which may use varyingcoding schemes. As the term is used herein, RAID 6 is defined as any N+2coding scheme which tolerates double disk failure, while user data iskept in the clear. This additional requirement assures that user readsare not affected by the RAID scheme under normal system operation.Examples of RAID 6 schemes include those that utilize the Reed Solomonerror correction code and those that utilize parity bits, such as thosewherein N data disks are supported by two redundancy disks each holdinga different parity bit. It should be noted that if all parity bits areon the same two disks, then the performance may be subject tobottlenecks. This can be solved by use of distributed parity stripesover N+2 disks similar to that specified in RAID 5. Examples of codingschemes based on parity calculations of rows and diagonals in a matrixof blocks include Even/Odd and Row Diagonal Parity (RDP). Both of theseschemes utilize a first parity disk “P” that holds the parities of rowsof blocks as well as a second parity disk “Q” that contains blocks thathold the parity of diagonals of data blocks. In both schemes, it isadvantageous to work with a block size that is smaller than the nativepage size. For example, the native page size may be 8 kilobytes (KB),while the block size is smaller but evenly divisible into 8 KB, e.g.,0.5 KB, 1 KB, 2 KB, 4 KB. In an example where the native page size is 8KB and the block size is 2 KB, each stripe thus may contain four rows,and thus the four blocks present on each disk form a single native page.However, a stripe can also be defined by multiple rows of blocksdistributed across the storage devices of the RAID array. It is assumedthat pages are read and written using a single disk operation.

An example RAID array includes five data disks denoted D0 through D4. Astorage controller (e.g., such as storage controller 108 or storagecontroller 208) is configured for writing initial data into the RAIDarray, and for updating existing data in the RAID array. The storagecontroller further provides functionality for recovering data aftersingle or double disk failure.

Each of the disks in the RAID array stores a column of data blocks. Thesame data block in successive disks forms a row, which is to say therows cross the disks. The data storage blocks are stored alongsideparity data blocks in parity disks denoted P and Q, and the numbers ofdata blocks in the different columns or disks may be different. Rowparity blocks are placed in a row parity column in disk P, and thediagonal parity data is placed in diagonal parity blocks in disk Q.

In the case of five data columns and four data rows, the number ofdiagonals is one greater than the number of rows. Thus, the diagonalparity column in disk Q includes one more block than the other columnsfor disks D0 through D4 and the row parity disk P.

The number of data columns is a prime number, and the number of rows isone less than that prime number (e.g., in the example the prime numberis 5, which corresponds to the five data disks D0 through D4). It shouldbe noted that, in practice, the various columns are distributed over theavailable physical disks to avoid system bottlenecks.

In one example distribution of data blocks in the RAID array, there arek data disks, where k=5 is a prime number, and there are five datacolumns corresponding to disks D0 through D4. There are four rows (e.g.,k−1). The P column includes the same four rows as the data columns D0through D4, but the Q column has an extra row. In one example, eachstripe is considered to contain k (where k must be prime) data columnsD0 through D4, and two parity columns P and Q. The stripe is composed ofa quasi-matrix of blocks, which contain k−1 rows. Column P contains k−1blocks, each providing the parity of the k data disk blocks in its row.The k by k−1 matrix made up of the blocks in the data columns includes kdiagonals each of size k−1. Column Q, in contrast with the rest of thecolumns, contains k blocks and not k−1. Each of the k blocks in disk Qholds the parity of one of the diagonals. It should be noted that theordering of blocks within each column may be arbitrary. Furthermore, theextra block in column Q may be placed in a data column which does notcontain a data block in the diagonal of which this block is the parity.Also, some of the rows may be blank.

It should be appreciated that there are various other ways to distributedata blocks in an array such as the example RAID array. For example, insome cases it may be desired to provide more than one row parity column,which results in higher capacity overhead but which allows for a fasterrebuild after a single disk failure.

Additional details regarding the above-described techniques for storingdata in RAID arrays are disclosed in U.S. Pat. No. 9,552,258, entitled“Method and System for Storing Data in RAID Memory Devices,” which isincorporated by reference herein.

In an illustrative embodiment, with reference now to FIG. 3, an examplestorage system 305 is illustrated that implements NVMeOF protocols. Insome embodiments, storage system 105 of FIG. 1 or storage system 205 ofFIG. 2 may comprise storage system 305 as illustrated in FIG. 3.

Storage system 305 comprises a plurality of storage enclosures 306,e.g., storage enclosures 306-1 through 306-N, one or more computeenclosures 308, e.g., compute enclosures 308-1 through 308-P, and anetwork fabric 309. In some embodiments, each storage enclosure 306 andcompute enclosure 308 may be physically located on a respective shelf ofthe storage system 305 and may communicate via the network fabric 309according to NVMeOF protocols.

Storage enclosures 306 comprise a RAID array 310, e.g., an array ofdisks 311-1 through 311-R, similar to RAID arrays 110 and 210. Disks311-1 through 311-R may comprise any storage device including, forexample, SSDs, platter drives, or any other storage device, similar tothe storage devices of RAID arrays 110 and 210. While described as aseparate RAID array 310 for each storage enclosure 306, in someembodiments the respective RAID arrays 310 of some or all of the storageenclosures 306 may function as a single RAID array 310.

Storage enclosures 306 also comprise one or more processing devices 312,e.g., processing devices 312-1 through 312-Q, similar to processingdevices 112 of FIG. 1 and 212 of FIG. 2, as described above. In someembodiments, each processing device 312 comprises one or more processors314 coupled to a memory 316. For example, processing device 312-1comprises a processor 314-1 coupled to a memory 316-1, processing device312-Q comprises a processor 314-Q coupled to a memory 316-Q, etc.

Processors 314 comprise any processor including, for example, amicroprocessor, a microcontroller, an application-specific integratedcircuit (ASIC), a field-programmable gate array (FPGA), graphicsprocessing unit (GPU) or other type of processing circuitry, as well asportions or combinations of such circuitry elements.

Memory 316 may comprise random access memory (RAM), read-only memory(ROM), flash memory or other types of memory, in any combination. Thememory and other memories disclosed herein should be viewed asillustrative examples of what are more generally referred to as“processor-readable storage media” storing executable program code ofone or more software programs.

Each processing device 312 of a given storage enclosure 306 isconfigured to communicate with the RAID array 310 of that storageenclosure 306 to perform various RAID processes, such as, e.g., RAIDparity calculations, RAID array recovery operations, or other similarRAID processes.

Compute enclosures 308 may, for example, comprise storage controllerssuch as, e.g., storage controller 108 or storage controller 208 and mayperform the functionality described herein in association with storagecontrollers of the storage systems 105, 205 and 305.

In some embodiments, storage enclosure 306 may also comprise a baseboardmanagement controller (BMC) 320. BMC 320 is configured to monitor thephysical state of the components of the storage enclosure 306,including, for example, processing devices 312 and to provide the stateto an administrator for the storage system 305.

In some embodiments, communication between the processing devices 312 ofeach storage enclosure 306 and between storage enclosures 306 andcompute enclosures 308 may be performed via a network fabric 309, e.g.,an NVMeOF network fabric. In some embodiments, network fabric 309 may bepart of network 104 of FIG. 1 or network 204 of FIG. 2. In someembodiments, network fabric 309 may be specific to storage system 305and separate from network 104 where, for example, network fabric 309provides communication channels between the various components ofstorage system 305.

While the following description may reference one of storage systems105, 205 and 305, the functionality described below may also oralternatively be implemented in any of the other storage systems 105,205 and 305.

In illustrative embodiments, RAID processing for a write flow may beoffloaded from the storage controller 108 to the processing devices 112of the storage enclosures 106 using write flow offload logic 114. Byoffloading the RAID processing to the processing devices 112, such as,e.g., calculating RAID parities, the processing resources of the storagecontroller 108 are freed up for use in processing additionalinput-output operations or other processing required by storage system105. The increased availability of the processing resources of thestorage controller 108 results in reduced system latency and increasedsystem throughput as the storage controller 108 no longer needs tocalculate RAID parities or perform other similar RAID processing.

In addition, because the RAID processing and calculation of RAIDparities are performed by the processing devices 112 of the storageenclosure 106 where that data will be stored, bandwidth usage betweenthe storage controller 108 and the RAID array 110 of the storageenclosure 106 is also reduced since the RAID parities or other similardata associated with the RAID processing does not need to be transferredbetween the storage controller 108 and the RAID array 110, therebyallowing the bandwidth to be utilized by the storage controller 108 forother data transfers, e.g., transferring more data pages, metadata, orother information data. For example, a RAID 6+2 scheme may typicallyrequire significant bandwidth for transferring RAID parities between thestorage controller 108 and the storage enclosure 106, e.g., up to 25% ormore of the bandwidth usage. In addition, if in-place upgrade RAIDtechniques are used instead of log-structured RAID techniques, thestorage controller 108 is required to retrieve the entire RAID stripefrom the disk array 110 for calculating the corresponding RAID paritybefore then sending the entire RAID stripe and calculated RAID parityback to the RAID array 110 for storage. Such a transfer of the RAIDstripe and the RAID parities incurs significant bandwidth usage that maybe better used for other input-output operations, for example, asdescribed above. In addition, such reading and transfer of the RAIDstripe may also result in substantial read amplification on the RAIDarray 110.

In some embodiments, the offloaded RAID processing for the write flowmay be performed by software executing on the processing devices 112 ofthe storage enclosure 106. In some embodiments, the processing devices112 may also or alternatively designate a hardware assisted offloadengine included in the enclosure 106 for performing the RAID processingfor the write flow. For example, a slot of the storage enclosure 106 maycomprise a hardware offload engine that is configured to perform some orall of the RAID processing offloaded from the storage controller 108 tothe processing devices 112 of that enclosure 106 or other enclosures106.

Illustrative embodiments of the techniques and functionality of writeflow offload logic 114 will now be described in more detail withreference to FIG. 4. The process of FIG. 4 is described with referencealso to FIGS. 1-3.

The process as shown in FIG. 4 includes steps 400 through 410, and issuitable for use in the system 100 but is more generally applicable toother types of systems comprising multiple host devices and a sharedstorage system. The shared storage system in this embodiment is assumedto comprise at least one storage array having a plurality of storagedevices. The storage devices can include logical storage devices, suchas LUNs or other logical storage volumes.

At 400, storage controller 108 obtains data pages associated with an IOrequest, for example, from a host device 102. For example, the datapages may be obtained by the storage controller 108 under the NVMeprotocol as a write command issued by the host device 102.

At 402, storage controller 108 provides the obtained data pages to theprocessing device 112 of a given storage enclosure 106 on which the datapages will be stored, e.g., via the network fabric 309 or anothercommunication channel.

At 404, storage controller 108 issues a command to the processing device112 of the given storage enclosure 106. For example, storage controller108 may issue a command instructing the processing device 112 to performone or more RAID operations such as, e.g., calculating RAID parity for astripe on which the obtained data pages will be stored. In someembodiments, the command includes an indication or identification of atarget stripe of RAID array 110 on which the obtained data pages will bestored.

For example, where a storage controller 108 will normally perform theRAID processing such as calculating RAID parity for data pages to bestored in a stripe itself, in the illustrative embodiments thisprocessing is offloaded to the processing device 112 of the storageenclosure 106 itself, thereby preserving processing resources of thestorage controller 108 for other uses.

At 406, the processing device 112 of the given storage enclosure 106receives the obtained data pages from the storage controller 108, e.g.,via the network fabric 309 or another communication channel.

At 408, responsive to receiving the command, the processing device 112of the given storage enclosure 106 calculates the RAID parities based atleast in part on the received data pages.

At 410, the processing device 112 stores the received data pages and thecalculated RAID parities on the RAID array 110 according to thearrangement of the RAID array 110, e.g., in the target stripe indicatedby the storage controller 108.

Once the storage controller 108 receives an indication that the datapages and parities have been written to the RAID array 110, the storagecontroller 108 hardens the new stripe layout and frees the associatedjournaling resources for those data pages in the storage system 105.

As described in the process of FIG. 4, once the storage controller sendsthe data pages and the command to the processing device 112, no furtheraction to process the data pages is required on the part of the storagecontroller 108. Instead, the processing device 112 of the storageenclosure 106 performs the required RAID processing to calculate theRAID parities and store the data pages and RAID parities together on theRAID array 110.

The use of write flow offload logic 114 provides substantial benefitsover utilizing the storage controller 108 to perform the RAID relatedprocessing during writes. For example, in the case of a 6+2 RAID scheme,up to 25% or more of the bandwidth on the data path between the storagecontroller 108 and the RAID array 110 may be saved by offloading theRAID calculations to the storage enclosure 106 since the RAID parities,and even the whole RAID stripe including the data pages (in the case ofan in-place upgrade RAID scheme) do not need to be transferred betweenthe storage controller 108 and the storage enclosure 106 which allowsmore resources to be available for use by storage controller 108 inservicing other IO requests.

In some embodiments, the data pages received by the processing device112 from the storage controller 108 are initially stored in thedesignated stripe of the RAID array 110 and are loaded into memory ofthe processing device 112, e.g., memory 316, from that stripe forcalculation of the RAID parities. Once the RAID parities are calculated,the RAID parities are then stored along with the already stored datapages according to the RAID arrangement of the RAID array 110.

In some embodiments, instead of initially storing the data pages in thedesignated stripe of RAID array 110, the data pages may instead beloaded directly into the memory of the processing device 112, e.g.,memory 316. The processing device 112 may then calculate the RAIDparities and store both the data pages and the RAID parities to the RAIDarray 110 in the designated stripe or a new stripe together, e.g., atthe same time or in close approximation to the same time. By initiallyloading the data pages directly into the memory of the processing device112 and not first storing them in the designated stripe of the RAIDarray 110, read amplification of the RAID array 110 may be avoided andan additional read of the data pages from the RAID array 110 to thememory of the processing device 112 just for parity calculations mayalso be avoided.

While the data pages are temporarily stored in the volatile memory ofthe processing device 112, and may be lost in the event of a systemreboot, drive failure, or other issue, the data pages are stillpreserved by the journaling mechanism of the cache 109 since the layoutof the stripe in the RAID array 110 is not hardened until the command iscompleted and the associated journal resources are released.

In illustrative embodiments, the processing for compression of datapages may be offloaded from the storage controller 108 to the processingdevices 112 of the storage enclosures 106 using compression offloadlogic 116. By offloading the compression of the data pages to theprocessing devices 112, the processing resources of the storagecontroller 108 are freed up for use in processing additional IOoperations or other processing required by storage system 105. Theincreased availability of the processing resources of the storagecontroller 108 results in reduced system latency and increased systemthroughput as the storage controller 108 no longer needs to perform thecompression of the data pages prior to storing the data pages in theRAID array 110.

In some embodiments, the offloaded compression processing may beperformed by software executing on the processing devices 112 of thestorage enclosure 106. In some embodiments, the processing devices 112may also or alternatively designate a hardware assisted offload engineincluded in the enclosure 106 for performing the compression processing.For example, a slot of the enclosure 106 may comprise a hardware offloadengine that is configured to perform some or all of the compressionprocessing offloaded from the storage controller 108 to the processingdevices 112 of that enclosure 106 or other enclosures 106.

Illustrative embodiments of the techniques and functionality ofcompression offload logic 116 will now be described in more detail withreference to FIG. 5. The process of FIG. 5 is described with referencealso to FIGS. 1-3.

The process as shown in FIG. 5 includes steps 500 through 512, and issuitable for use in the system 100 but is more generally applicable toother types of systems comprising multiple host devices and a sharedstorage system. The shared storage system in this embodiment is assumedto comprise at least one storage array having a plurality of storagedevices. The storage devices can include logical storage devices, suchas LUNs or other logical storage volumes.

At 500, storage controller 108 obtains data pages associated with an IOrequest, for example, from a host device 102. For example, the datapages may be obtained by the storage controller 108 under the NVMeprotocol as a write command issued by the host device 102.

At 502, storage controller 108 provides the obtained data pages to theprocessing device 112 of a given storage enclosure 106 on which the datapages will be stored, e.g., via the network fabric 309 or anothercommunication channel.

At 504, storage controller 108 issues a command to the processing device112 of the given storage enclosure 106. For example, storage controller108 may issue a command instructing the processing device 112 to performcompression on the obtained data pages. In some embodiments, the commandmay also instruct the processing device 112 to perform one or more RAIDoperations such as, e.g., calculating RAID parity for a stripe on whichcompressed data pages will be stored. In some embodiments, the commandincludes an indication or identification of a target stripe of RAIDarray 110 on which the compressed data pages will be stored.

For example, where a storage controller 108 will normally perform thecompression of data pages to be stored in a stripe by itself, in theillustrative embodiments this processing is offloaded to the processingdevice 112 of the storage enclosure 106, thereby preserving processingresources of the storage controller 108 for other uses.

At 506, the processing device 112 of the given storage enclosure 106receives the obtained data pages from the storage controller 108, e.g.,via the network fabric 309 or another communication channel.

At 508, responsive to receiving the command, the processing device 112of the given storage enclosure 106 generates compressed data pages basedat least in part on the received data pages. In some embodiments, theprocessing device 112 may also calculate RAID parities based at least inpart on one or more of the compressed data pages.

At 510, the processing device 112 stores one or more of the compresseddata pages and the calculated RAID parities on the RAID array 110according to the arrangement of the RAID array 110, e.g., in the targetstripe indicated by the storage controller 108.

At 512, the processing device 112 returns information, e.g., tokens,associated with the storage of the one or more compressed data pages tothe storage controller 108. For example, the information may comprisetokens that indicate a location of the one or more compressed data pageson a stripe of the RAID array 110, an offset into the stripe, acompression ratio that was used in the compression, the compressionalgorithm, or other similar information.

In some embodiments, not all of the compressed data pages will fit onthe target stripe. In such a circumstance, the information may alsocomprise a token or other indication of what data pages were not storedin the stripe. For example, the token may indicate a size of thecompressed data pages that were not stored in the stripe.

In some embodiments, for example, where some or all of the data pagesare not compressible, e.g., data pages comprising encrypted data orother similar uncompressible data, the information may also comprise atoken or other indication of the data pages that were not compressible.

Once the storage controller 108 receives an indication that the one ormore compressed data pages and associated parities have been written tothe RAID array 110, the storage controller 108 hardens the new stripelayout and frees the associated journaling resources for those datapages in the storage system 105.

As described in the process of FIG. 5, once the storage controller sendsthe data pages and the command to the processing device 112, no furtheraction to process the data pages is required on the part of the storagecontroller 108. Instead, the processing device 112 of the storageenclosure 106 performs the required compression of the data pages andcalculation of the associated RAID parities and stores the compresseddata pages and RAID parities together on the RAID array 110.

Because the storage controller 108 does not know what compression willbe used on the data pages and the size of the data pages that isrequired to fill the RAID stripe, the storage controller 108 may, inillustrative embodiments, estimate the data pages that will be requiredto fill a RAID stripe on the RAID array 110. For example, in someembodiments, the storage controller 108 may send a set of data pageshaving a size that corresponds to an average or maximum compressionratio sufficient to fill a stripe to the processing device 112.

In an example scenario, where the size of the data pages sent by storagecontroller 108 to the processing device 112 is greater than the size ofthe target stripe after compression has been performed, processingdevice 112 may indicate to the storage controller 108 which extracompressed data pages, their size, or other similar information, werenot stored on the stripe. The storage controller 108 may use thisinformation when estimating the size of the next set of data pages tosend to the processing device 112, for example, by reducing the size ofthe next set of data pages relative to the average or maximumcompression ratio based at least in part on the presences of the extracompressed data pages that still need to be added to a stripe. The nextset of data pages is then compressed by processing device 112 and someor all of those compressed data pages are stored in a stripe of the RAIDarray 110 along with the extra compressed data pages. The processingdevice 112 may then once again indicate to the storage controller 108which, if any, of the newly compressed data pages do not fit on the newstripe.

The use of compression offload logic 116 provides substantial benefitsover utilizing the storage controller 108 to perform the compressionduring writes. For example, by offloading the compression to theprocessing device 112, more resources are available for use by storagecontroller 108 in servicing other IO requests.

In some embodiments, when an IO request to read a particular data pagethat is stored in compressed form on RAID array 110 is received by thestorage controller 108, e.g., from a host device 102, the storagecontroller 108 may issue a command to the processing device 112 of agiven storage enclosure 106 to retrieve the particular data page. Inresponse to receiving the command, the processing device 112 mayretrieve the corresponding compressed data page, decompress the datapage, and provide the decompressed data page to the storage controller108.

In some embodiments, processing device 112 may alternatively provide thecompressed data page to the storage controller 108, i.e., withoutdecompressing the data page, and the storage controller 108 maydecompress the data page. This embodiment may reduce the requiredbandwidth to provide the data page from the storage enclosure 106 to thestorage controller 108 since the data page is still compressed. Theusage of processing resources is relatively minor as the decompressionoperations are not CPU intensive.

In some embodiments, for example, where a disk failure or other failurehas occurred, the rebuilding of the RAID array 110 may also be offloadedto the processing device 112 of the storage enclosure 106 from thestorage controller 108. For example, the processing device 112 mayperform the RAID processes required to rebuild the disk, e.g., byreading the data pages and parities from the disks and calculating theneeded data pages based at least in part on the parities. The rebuiltdata pages may then be provided to the storage controller 108 by theprocessing device 112, in either compressed or uncompressed form, asdescribed above.

While described as separate embodiments above, in some embodiments, thefunctionality implemented by the write flow offload logic 114 andcompression offload logic 116 may be utilized together.

It is to be understood that for any methodologies described herein,e.g., write flow and compression offloading, the ordering of the processsteps may be varied in other embodiments, or certain steps may beperformed at least in part concurrently with one another rather thanserially. Also, one or more of the process steps may be repeatedperiodically, or multiple instances of the process can be performed inparallel with one another in order to implement a plurality of differentprocesses for different storage systems or for different RAID arrays orother data striping schemes on a particular storage system or systems.

Functionality such as that described herein can be implemented at leastin part in the form of one or more software programs stored in memoryand executed by a processor of a processing device such as a computer orserver. As will be described below, a memory or other storage devicehaving executable program code of one or more software programs embodiedtherein is an example of what is more generally referred to herein as a“processor-readable storage medium.”

For example, a host device such as host device 102 or a storagecontroller such as storage controller 208 that is configured to controlperformance of one or more steps described herein can be implemented aspart of what is more generally referred to herein as a processingplatform comprising one or more processing devices each comprising aprocessor coupled to a memory. Such processing devices are to bedistinguished from processing devices referred to herein with respect tothe processing capabilities of the SSDs. In the case of a host device orstorage controller, a given such processing device may correspond to oneor more virtual machines or other types of virtualization infrastructuresuch as Docker containers or Linux containers (LXCs). The host device102 of system 100 or the storage controller 208 of system 200, as wellas other system components, may be implemented at least in part usingprocessing devices of such processing platforms. For example, in adistributed implementation of the storage controller 208, respectivedistributed modules of such a storage controller can be implemented inrespective containers running on respective ones of the processingdevices of a processing platform.

In some embodiments, the storage system comprises an XtremIO™ storagearray or other type of content addressable storage system suitablymodified to incorporate functionality for write flow and compressionoffloading as disclosed herein.

An illustrative embodiment of such a content addressable storage systemwill now be described with reference to FIG. 6. In this embodiment, acontent addressable storage system 605 comprises a plurality of storagedevices 606, an associated storage controller 608, and an associatedcache 609. The content addressable storage system 605 may be viewed as aparticular implementation of the storage system 205, and accordingly isassumed to be coupled to host devices 202 of computer system 201 vianetwork 204 within information processing system 200.

The storage controller 608 in the present embodiment is configured toimplement functionality for write flow and compression offloading of thetype previously described in conjunction with FIGS. 1 through 5.

The storage controller 608 includes write flow offload logic 614, whichis configured to operate in a manner similar to that described above forrespective corresponding write flow offload logic 114 and 214, andincludes compression offload logic 616, which is configured to operatein a manner similar to that described above for respective correspondingcompression offload logic 116 and 216.

The cache 609 is configured to operate in a manner similar to thatdescribed above for respective cache 109 and 209.

The content addressable storage system 605 in the FIG. 6 embodiment isimplemented as at least a portion of a clustered storage system andincludes a plurality of storage nodes 615 each comprising acorresponding subset of the storage devices 606. Other clustered storagesystem arrangements comprising multiple storage nodes can be used inother embodiments. A given clustered storage system may include not onlystorage nodes 615 but also additional storage nodes coupled via astorage network. Alternatively, such additional storage nodes may bepart of another clustered storage system of the system 200. Each of thestorage nodes 615 of the storage system 605 is assumed to be implementedusing at least one processing device comprising a processor coupled to amemory.

The storage controller 608 of the content addressable storage system 605is implemented in a distributed manner so as to comprise a plurality ofdistributed storage controller components implemented on respective onesof the storage nodes 615. The storage controller 608 is therefore anexample of what is more generally referred to herein as a “distributedstorage controller.” In subsequent description herein, the storagecontroller 608 may be more particularly referred to as a distributedstorage controller.

Each of the storage nodes 615 in this embodiment further comprises a setof processing modules configured to communicate over one or morenetworks with corresponding sets of processing modules on other ones ofthe storage nodes 615. The sets of processing modules of the storagenodes 615 collectively comprise at least a portion of the distributedstorage controller 608 of the content addressable storage system 605.

The modules of the distributed storage controller 608 in the presentembodiment more particularly comprise different sets of processingmodules implemented on each of the storage nodes 615. The set ofprocessing modules of each of the storage nodes 615 comprises at least acontrol module 608C, a data module 608D and a routing module 608R. Thedistributed storage controller 608 further comprises one or moremanagement (“MGMT”) modules 608M. For example, only a single one of thestorage nodes 615 may include a management module 608M. It is alsopossible that management modules 608M may be implemented on each of atleast a subset of the storage nodes 615.

Each of the storage nodes 615 of the storage system 605 thereforecomprises a set of processing modules configured to communicate over oneor more networks with corresponding sets of processing modules on otherones of the storage nodes. A given such set of processing modulesimplemented on a particular storage node illustratively includes atleast one control module 608C, at least one data module 608D and atleast one routing module 608R, and possibly a management module 608M.These sets of processing modules of the storage nodes collectivelycomprise at least a portion of the distributed storage controller 608.

Communication links may be established between the various processingmodules of the distributed storage controller 608 using well-knowncommunication protocols such as IP, Transmission Control Protocol (TCP),and remote direct memory access (RDMA). For example, respective sets ofIP links used in data transfer and corresponding messaging could beassociated with respective different ones of the routing modules 608R.

Although shown as a separate logic of the distributed storage controller608, the write flow offload logic 614 and compression offload logic 616in the present embodiment are assumed to be distributed at least in partover at least a subset of the other modules 608C, 608D, 608R and 608M ofthe storage controller 608. Accordingly, at least portions of thefunctionality of write flow offload logic 614 and compression offloadlogic 616 may be implemented in one or more of the other modules of thestorage controller 608. In other embodiments, the write flow offloadlogic 614 and compression offload logic 616 may be implemented as astand-alone module of the storage controller 608.

The storage devices 606 are configured to store metadata pages 620 anduser data pages 622 and may also store additional information notexplicitly shown such as checkpoints and write journals. The metadatapages 620 and the user data pages 622 are illustratively stored inrespective designated metadata and user data areas of the storagedevices 606. Accordingly, metadata pages 620 and user data pages 622 maybe viewed as corresponding to respective designated metadata and userdata areas of the storage devices 606.

A given “page” as the term is broadly used herein should not be viewedas being limited to any particular range of fixed sizes. In someembodiments, a page size of 8 KB is used, but this is by way of exampleonly and can be varied in other embodiments. For example, page sizes of4 KB, 16 KB or other values can be used. Accordingly, illustrativeembodiments can utilize any of a wide variety of alternative pagingarrangements for organizing the metadata pages 620 and the user datapages 622.

The user data pages 622 are part of a plurality of logical units (LUNs)configured to store files, blocks, objects or other arrangements ofdata, each also generally referred to herein as a “data item,” on behalfof users associated with host devices 202. Each such LUN may compriseparticular ones of the above-noted pages of the user data area. The userdata stored in the user data pages 622 can include any type of user datathat may be utilized in the system 200. The term “user data” herein istherefore also intended to be broadly construed.

The content addressable storage system 605 in the embodiment of FIG. 6is configured to generate hash metadata providing a mapping betweencontent-based digests of respective ones of the user data pages 622 andcorresponding physical locations of those pages in the user data area.Content-based digests generated using hash functions are also referredto herein as “hash digests.” The hash metadata generated by the contentaddressable storage system 605 is illustratively stored as metadatapages 620 in the metadata area. The generation and storage of the hashmetadata is assumed to be performed under the control of the storagecontroller 608.

Each of the metadata pages 620 characterizes a plurality of the userdata pages 622. For example, a given set of user data pages representinga portion of the user data pages 622 illustratively comprises aplurality of user data pages denoted User Data Page 1, User Data Page 2,. . . User Data Page n. It should be noted that usage of the variable nin this user data page context is unrelated to its usage elsewhereherein.

Each of the user data pages 622 in this example is characterized by aLUN identifier, an offset and a content-based signature. Thecontent-based signature is generated as a hash function of content ofthe corresponding user data page. Illustrative hash functions that maybe used to generate the content-based signature include the above-notedSHA1 hash function, or other secure hashing algorithms known to thoseskilled in the art. The content-based signature is utilized to determinethe location of the corresponding user data page within the user dataarea of the storage devices 606.

Each of the metadata pages 620 in the present embodiment is assumed tohave a signature that is not content-based. For example, the metadatapage signatures may be generated using hash functions or other signaturegeneration algorithms that do not utilize content of the metadata pagesas input to the signature generation algorithm. Also, each of themetadata pages is assumed to characterize a different set of the userdata pages.

A given set of metadata pages representing a portion of the metadatapages 620 in an illustrative embodiment comprises metadata pages denotedMetadata Page 1, Metadata Page 2, . . . Metadata Page m, havingrespective signatures denoted Signature 1, Signature 2, . . . Signaturem. Each such metadata page characterizes a different set of n user datapages. For example, the characterizing information in each metadata pagecan include the LUN identifiers, offsets and content-based signaturesfor each of the n user data pages that are characterized by thatmetadata page. It is to be appreciated, however, that the user data andmetadata page configurations described above are examples only, andnumerous alternative user data and metadata page configurations can beused in other embodiments.

Ownership of a user data logical address space within the contentaddressable storage system 605 is illustratively distributed among thecontrol modules 608C.

The functionality provided by write flow offload logic 614 andcompression offload logic 616 in this embodiment is assumed to bedistributed across multiple distributed processing modules, including atleast a subset of the processing modules 608C, 608D, 608R and 608M ofthe distributed storage controller 608.

For example, the management module 608M of the storage controller 608may include logic that engages corresponding logic instances in all ofthe control modules 608C and routing modules 608R in order to implementprocesses for write flow offloading and compression offloading.

In some embodiments, the content addressable storage system 605comprises an XtremIO™ storage array suitably modified to incorporatetechniques for write flow offloading and compression offloading, asdisclosed herein.

In arrangements of this type, the control modules 608C, data modules608D and routing modules 608R of the distributed storage controller 608illustratively comprise respective C-modules, D-modules and R-modules ofthe XtremIO™ storage array. The one or more management modules 608M ofthe distributed storage controller 608 in such arrangementsillustratively comprise a system-wide management module (“SYM module”)of the XtremIO™ storage array, although other types and arrangements ofsystem-wide management modules can be used in other embodiments.Accordingly, functionality for write flow offloading and compressionoffloading in some embodiments is implemented under the control of atleast one system-wide management module of the distributed storagecontroller 608, utilizing the C-modules, D-modules and R-modules of theXtremIO™ storage array.

In the above-described XtremIO™ storage array example, each user datapage has a fixed size such as 8 KB and its content-based signature is a20-byte signature generated using an SHA1 hash function. Also, each pagehas a LUN identifier and an offset, and so is characterized by <lun_id,offset, signature>.

The content-based signature in the present example comprises acontent-based digest of the corresponding data page. Such acontent-based digest is more particularly referred to as a “hash digest”of the corresponding data page, as the content-based signature isillustratively generated by applying a hash function such as SHA1 to thecontent of that data page. The full hash digest of a given data page isgiven by the above-noted 20-byte signature. The hash digest may berepresented by a corresponding “hash handle,” which in some cases maycomprise a particular portion of the hash digest. The hash handleillustratively maps on a one-to-one basis to the corresponding full hashdigest within a designated cluster boundary or other specified storageresource boundary of a given storage system. In arrangements of thistype, the hash handle provides a lightweight mechanism for uniquelyidentifying the corresponding full hash digest and its associated datapage within the specified storage resource boundary. The hash digest andhash handle are both considered examples of “content-based signatures”as that term is broadly used herein.

Examples of techniques for generating and processing hash handles forrespective hash digests of respective data pages are disclosed in U.S.Pat. No. 9,208,162, entitled “Generating a Short Hash Handle,” and U.S.Pat. No. 9,286,003, entitled “Method and Apparatus for Creating a ShortHash Handle Highly Correlated with a Globally-Unique Hash Signature,”both of which are incorporated by reference herein.

As mentioned previously, storage controller components in an XtremIO™storage array illustratively include C-module, D-module and R-modulecomponents. For example, separate instances of such components can beassociated with each of a plurality of storage nodes in a clusteredstorage system implementation.

The distributed storage controller in this example is configured togroup consecutive pages into page groups, to arrange the page groupsinto slices, and to assign the slices to different ones of theC-modules. For example, if there are 624 slices distributed evenlyacross the C-modules, and there are a total of 16 C-modules in a givenimplementation, each of the C-modules “owns” 1024/16=64 slices. In sucharrangements, different ones of the slices are assigned to differentones of the control modules 608C such that control of the slices withinthe storage controller 608 of the storage system 605 is substantiallyevenly distributed over the control modules 608C of the storagecontroller 608.

The D-module allows a user to locate a given user data page based on itssignature. Each metadata page also has a size of 8 KB and includesmultiple instances of the <lun_id, offset, signature>for respective onesof a plurality of the user data pages. Such metadata pages areillustratively generated by the C-module but are accessed using theD-module based on a metadata page signature.

The metadata page signature in this embodiment is a 20-byte signaturebut is not based on the content of the metadata page. Instead, themetadata page signature is generated based on an 8-byte metadata pageidentifier that is a function of the LUN identifier and offsetinformation of that metadata page.

If a user wants to read a user data page having a particular LUNidentifier and offset, the corresponding metadata page identifier isfirst determined, then the metadata page signature is computed for theidentified metadata page, and then the metadata page is read using thecomputed signature. In this embodiment, the metadata page signature ismore particularly computed using a signature generation algorithm thatgenerates the signature to include a hash of the 8-byte metadata pageidentifier, one or more ASCII codes for particular predeterminedcharacters, as well as possible additional fields. The last bit of themetadata page signature may always be set to a particular logic value soas to distinguish it from the user data page signature in which the lastbit may always be set to the opposite logic value.

The metadata page signature is used to retrieve the metadata page viathe D-module. This metadata page will include the <lun_id, offset,signature>for the user data page if the user page exists. The signatureof the user data page is then used to retrieve that user data page, alsovia the D-module.

Write requests processed in the content addressable storage system 605each illustratively comprise one or more IO operations directing that atleast one data item of the storage system 605 be written to in aparticular manner. A given write request is illustratively received inthe storage system 605 from a host device, illustratively one of thehost devices 202. In some embodiments, a write request is received inthe distributed storage controller 608 of the storage system 605 anddirected from one processing module to another processing module of thedistributed storage controller 608. For example, a received writerequest may be directed from a routing module 608R of the distributedstorage controller 608 to a particular control module 608C of thedistributed storage controller 608. Other arrangements for receiving andprocessing write requests from one or more host devices can be used.

The term “write request” as used herein is intended to be broadlyconstrued, so as to encompass one or more IO operations directing thatat least one data item of a storage system be written to in a particularmanner. A given write request is illustratively received in a storagesystem from a host device.

In the XtremIO™ context, the C-modules, D-modules and R-modules of thestorage nodes 615 communicate with one another over a high-speedinternal network such as an InfiniBand network. The C-modules, D-modulesand R-modules coordinate with one another to accomplish various IOprocessing tasks.

The write requests from the host devices identify particular data pagesto be written in the storage system 605 by their corresponding logicaladdresses each comprising a LUN ID and an offset.

As noted above, a given one of the content-based signaturesillustratively comprises a hash digest of the corresponding data page,with the hash digest being generated by applying a hash function to thecontent of that data page. The hash digest may be uniquely representedwithin a given storage resource boundary by a corresponding hash handle.

The storage system 605 utilizes a two-level mapping process to maplogical block addresses to physical block addresses. The first level ofmapping uses an address-to-hash (“A2H”) table and the second level ofmapping uses a hash metadata (“HMD”) table, with the A2H and HMD tablescorresponding to respective logical and physical layers of thecontent-based signature mapping within the storage system 605.

The first level of mapping using the A2H table associates logicaladdresses of respective data pages with respective content-basedsignatures of those data pages. This is also referred to logical layermapping.

The second level of mapping using the HMD table associates respectiveones of the content-based signatures with respective physical storagelocations in one or more of the storage devices 606. This is alsoreferred to as physical layer mapping.

For a given write request, both of the corresponding HMD and A2H tablesare updated in conjunction with the processing of that write request.

The A2H and HMD tables described above are examples of what are moregenerally referred to herein as “mapping tables” of respective first andsecond distinct types. Other types and arrangements of mapping tables orother content-based signature mapping information may be used in otherembodiments.

The logical block addresses or LBAs of a logical layer of the storagesystem 605 correspond to respective physical blocks of a physical layerof the storage system 605. The user data pages of the logical layer areorganized by LBA and have reference via respective content-basedsignatures to particular physical blocks of the physical layer.

Each of the physical blocks has an associated reference count that ismaintained within the storage system 605. The reference count for agiven physical block indicates the number of logical blocks that pointto that same physical block.

In releasing logical address space in the storage system, adereferencing operation is generally executed for each of the LBAs beingreleased. More particularly, the reference count of the correspondingphysical block is decremented. A reference count of zero indicates thatthere are no longer any logical blocks that reference the correspondingphysical block, and so that physical block can be released.

It should also be understood that the particular arrangement of storagecontroller processing modules 608C, 608D, 608R and 608M as shown in theFIG. 6 embodiment is presented by way of example only. Numerousalternative arrangements of processing modules of a distributed storagecontroller may be used to implement functionality for determiningcompression block size and selecting prime numbers and associatednumbers of sub-stripes for efficient packing of compressed data in aclustered storage system in other embodiments.

Additional examples of content addressable storage functionalityimplemented in some embodiments by control modules 608C, data modules608D, routing modules 608R and management module(s) 608M of distributedstorage controller 608 can be found in U.S. Pat. No. 9,104,326, entitled“Scalable Block Data Storage Using Content Addressing,” which isincorporated by reference herein. Alternative arrangements of these andother storage node processing modules of a distributed storagecontroller in a content addressable storage system can be used in otherembodiments.

Illustrative embodiments of host devices or storage systems withfunctionality for write flow offloading and compression offloading canprovide a number of significant advantages relative to conventionalarrangements. For example, some embodiments provide techniques for writeflow offloading and compression offloading that reduce the processingthat is required to be performed by the storage controller and reducethe amount of bandwidth usage between the storage controller and theRAID array. These techniques allow the storage controller to free upprocessing resources and bandwidth for use in servicing additional IOrequests or other system needs.

It is to be appreciated that the particular advantages described aboveand elsewhere herein are associated with particular illustrativeembodiments and need not be present in other embodiments. Also, theparticular types of information processing system features andfunctionality as illustrated in the drawings and described above areexemplary only, and numerous other arrangements may be used in otherembodiments.

Illustrative embodiments of processing platforms utilized to implementhost devices and storage systems with functionality for write flowoffloading and compression offloading will now be described in greaterdetail with reference to FIGS. 7 and 8. Although described in thecontext of system 100, these platforms may also be used to implement atleast portions of other information processing systems in otherembodiments.

FIG. 7 shows an example processing platform comprising cloudinfrastructure 700. The cloud infrastructure 700 comprises a combinationof physical and virtual processing resources that may be utilized toimplement at least a portion of the information processing system 100.The cloud infrastructure 700 comprises multiple virtual machines (VMs)and/or container sets 702-1, 702-2, . . . 702-L implemented usingvirtualization infrastructure 704. The virtualization infrastructure 704runs on physical infrastructure 705, and illustratively comprises one ormore hypervisors and/or operating system level virtualizationinfrastructure. The operating system level virtualization infrastructureillustratively comprises kernel control groups of a Linux operatingsystem or other type of operating system.

The cloud infrastructure 700 further comprises sets of applications710-1, 710-2, . . . 710-L running on respective ones of theVMs/container sets 702-1, 702-2, . . . 702-L under the control of thevirtualization infrastructure 704. The VMs/container sets 702 maycomprise respective VMs, respective sets of one or more containers, orrespective sets of one or more containers running in VMs.

In some implementations of the FIG. 7 embodiment, the VMs/container sets702 comprise respective VMs implemented using virtualizationinfrastructure 704 that comprises at least one hypervisor. Suchimplementations can provide functionality for write flow offloading andcompression offloading of the type described above for one or moreprocesses running on a given one of the VMs. For example, each of theVMs can implement such functionality for one or more processes runningon that particular VM.

An example of a hypervisor platform that may be used to implement ahypervisor within the virtualization infrastructure 704 is the VMware®vSphere® which may have an associated virtual infrastructure managementsystem such as the VMware® vCenter™. The underlying physical machinesmay comprise one or more distributed processing platforms that includeone or more storage systems.

In other implementations of the FIG. 7 embodiment, the VMs/containersets 702 comprise respective containers implemented using virtualizationinfrastructure 704 that provides operating system level virtualizationfunctionality, such as support for Docker containers running on baremetal hosts, or Docker containers running on VMs. The containers areillustratively implemented using respective kernel control groups of theoperating system. Such implementations can provide functionality forwrite flow offloading and compression offloading of the type describedabove for one or more processes running on different ones of thecontainers. For example, a container host device supporting multiplecontainers of one or more container sets can implement one or moreinstances of such functionality or logic.

As is apparent from the above, one or more of the processing modules orother components of system 100 may each run on a computer, server,storage device or other processing platform element. A given suchelement may be viewed as an example of what is more generally referredto herein as a “processing device.” The cloud infrastructure 700 shownin FIG. 7 may represent at least a portion of one processing platform.Another example of such a processing platform is processing platform 800shown in FIG. 8.

The processing platform 800 in this embodiment comprises a portion ofsystem 100 or 200 and includes a plurality of processing devices,denoted 802-1, 802-2, 802-3, . . . 802-K, which communicate with oneanother over a network 804.

The network 804 may comprise any type of network, including by way ofexample a global computer network such as the Internet, a WAN, a LAN, asatellite network, a telephone or cable network, a cellular network, awireless network such as a WiFi or WiMAX network, or various portions orcombinations of these and other types of networks.

The processing device 802-1 in the processing platform 800 comprises aprocessor 810 coupled to a memory 812.

The processor 810 may comprise a microprocessor, a microcontroller, anapplication-specific integrated circuit (ASIC), a field-programmablegate array (FPGA) or other type of processing circuitry, as well asportions or combinations of such circuitry elements.

The memory 812 may comprise random access memory (RAM), read-only memory(ROM), flash memory or other types of memory, in any combination. Thememory 812 and other memories disclosed herein should be viewed asillustrative examples of what are more generally referred to as“processor-readable storage media” storing executable program code ofone or more software programs.

Articles of manufacture comprising such processor-readable storage mediaare considered illustrative embodiments. A given such article ofmanufacture may comprise, for example, a storage array, a storage diskor an integrated circuit containing RAM, ROM, flash memory or otherelectronic memory, or any of a wide variety of other types of computerprogram products. The term “article of manufacture” as used hereinshould be understood to exclude transitory, propagating signals.Numerous other types of computer program products comprisingprocessor-readable storage media can be used.

Also included in the processing device 802-1 is network interfacecircuitry 814, which is used to interface the processing device with thenetwork 804 and other system components and may comprise conventionaltransceivers.

The other processing devices 802 of the processing platform 800 areassumed to be configured in a manner similar to that shown forprocessing device 802-1 in the figure.

Again, the particular processing platform 800 shown in the figure ispresented by way of example only, and system 100 or 200 may includeadditional or alternative processing platforms, as well as numerousdistinct processing platforms in any combination, with each suchplatform comprising one or more computers, servers, storage devices orother processing devices.

For example, other processing platforms used to implement illustrativeembodiments can comprise converged infrastructure such as VxRail™,VxRack™, VxRack™ FLEX, VxBlock™ or Vblock® converged infrastructure fromVCE, the Virtual Computing Environment Company, now the ConvergedPlatform and Solutions Division of Dell EMC.

It should therefore be understood that in other embodiments differentarrangements of additional or alternative elements may be used. At leasta subset of these elements may be collectively implemented on a commonprocessing platform, or each such element may be implemented on aseparate processing platform.

As indicated previously, components of an information processing systemas disclosed herein can be implemented at least in part in the form ofone or more software programs stored in memory and executed by aprocessor of a processing device. For example, at least portions of thefunctionality for determining compression block size and selecting primenumbers and associated numbers of sub-stripes for efficient packing ofcompressed data as disclosed herein are illustratively implemented inthe form of software running on one or more processing devices.

It should again be emphasized that the above-described embodiments arepresented for purposes of illustration only. Many variations and otheralternative embodiments may be used. For example, the disclosedtechniques are applicable to a wide variety of other types ofinformation processing systems, host devices, storage systems, storagenodes, storage devices, storage controllers, RAID arrays or other datastriping, etc. Also, the particular configurations of system and deviceelements and associated processing operations illustratively shown inthe drawings can be varied in other embodiments. Moreover, the variousassumptions made above in the course of describing the illustrativeembodiments should also be viewed as exemplary rather than asrequirements or limitations of the disclosure. Numerous otheralternative embodiments within the scope of the appended claims will bereadily apparent to those skilled in the art.

What is claimed is:
 1. An apparatus comprising: a storage systemcomprising a plurality of enclosures and a storage controller, eachenclosure comprising at least one processing device coupled to memoryand a redundant array of independent disks (RAID) arrangement comprisinga plurality of drives, the at least one processing device being separatefrom the RAID arrangement; the storage controller configured: to obtaindata pages associated with at least one input-output request; to providethe obtained data pages to the at least one processing device of a givenenclosure of the plurality of enclosures; and to issue a command to theat least one processing device of the given enclosure to perform atleast one operation based at least in part on the obtained data pages;the at least one processing device of the given enclosure configured: toreceive the obtained data pages from the storage controller; responsiveto receiving the command from the storage controller, to generatecompressed data pages based at least in part on the received data pages;to store one or more of the compressed data pages on a stripe of theplurality of drives according to the RAID arrangement; to determine thatat least one other compressed data page of the compressed data pagesdoes not fit on the stripe of the plurality of drives and has not beenstored on the plurality of drives; and to return information associatedwith the storage of the one or more of the compressed data pages to thestorage controller, the information comprising an indication that the atleast one other compressed data page does not fit on the stripe of theplurality of drives and has not been stored on the plurality of drivesand a size of the at least one other compressed data page, the storagecontroller being configured to utilize the information to access the oneor more of the compressed data pages stored on the plurality of drives;wherein the storage controller is further configured: to obtainadditional data pages associated with another input-output request; todetermine an estimated size of data pages that when compressed by the atleast one processing device of the given enclosure would fill a stripe;to reduce the estimated size based at least in part on the size of theat least one other compressed data page found in the information; toselect one or more data pages of the additional data pages based atleast in part on the reduced estimated size; and to provide the selectedone or more data pages of the additional data pages to the at least oneprocessing device of the given enclosure.
 2. The apparatus of claim 1wherein the information comprises an indication of a location of each ofthe one or more compressed data pages that was stored on the pluralityof drives and a size of each of the one or more compressed data pagesthat was stored on the plurality of drives.
 3. The apparatus of claim 1wherein generating the compressed data pages based at least in part onthe received data pages comprises at least one of: causing a compressionoffload engine of the storage system to compress the received datapages; and compressing the received data pages by the at least oneprocessing device of the given enclosure.
 4. The apparatus of claim 1wherein responsive to receiving a read command from the storagecontroller that targets a compressed data page stored in the pluralityof drives, the at least one processing device of the given enclosure isconfigured: to retrieve the targeted compressed data page from theplurality of drives; to decompress the targeted compressed data page togenerate a decompressed data page; and to provide the decompressed datapage to the storage controller.
 5. The apparatus of claim 1 whereinresponsive to receiving a read command from the storage controller thattargets a compressed data page stored in the plurality of drives, the atleast one processing device of the given enclosure is configured: toretrieve the targeted compressed data page from the plurality of drives;and to provide the targeted compressed data page to the storagecontroller; wherein responsive to receiving targeted compressed datapage from the at least one processing device of the given enclosure, thestorage controller is configured: to receive the targeted compresseddata page from the at least one processing device of the givenenclosure; and to decompress the targeted compressed data page.
 6. Theapparatus of claim 1 wherein responsive an indication of a disk failure,the at least one processing device of the given enclosure is configured:to read RAID parities from the plurality of drives; to regenerate theone or more compressed data pages based at least in part on the RAIDparities; to decompress the regenerated one or more compressed datapages; and to provide the one or more decompressed data pages to thestorage controller.
 7. A method comprising: obtaining, by a storagecontroller of a storage system, data pages associated with at least oneinput-output request, the storage system comprising a plurality ofenclosures and the storage controller, each enclosure comprising atleast one processing device coupled to memory and a redundant array ofindependent disks (RAID) arrangement comprising a plurality of drives,the at least one processing device being separate from the RAIDarrangement; providing, by the storage controller, the obtained datapages to the at least one processing device of a given enclosure of theplurality of enclosures; issuing, by the storage controller, a commandto the at least one processing device of the given enclosure to performat least one operation based at least in part on the obtained datapages; receiving, by the at least one processing device of the givenenclosure, the obtained data pages from the storage controller;responsive to receiving the command from the storage controller,generating, by the at least one processing device of the givenenclosure, compressed data pages based at least in part on the receiveddata pages; storing, by the at least one processing device of the givenenclosure, one or more of the compressed data pages on the plurality ofdrives according to the RAID arrangement; determining, by the at leastone processing device of the given enclosure, that at least one othercompressed data page of the compressed data pages does not fit on thestripe of the plurality of drives and has not been stored on theplurality of drives; returning, by the at least one processing device ofthe given enclosure, information associated with the storage of the oneor more of the compressed data pages to the storage controller, theinformation comprising an indication that the at least one othercompressed data page does not fit on the stripe of the plurality ofdrives and has not been stored on the plurality of drives and a size ofthe at least one other compressed data page, the storage controllerbeing configured to utilize the information to access the one or more ofthe compressed data pages stored on the plurality of drives; obtaining,by the storage controller, additional data pages associated with anotherinput-output request; determining, by the storage controller, anestimated size of data pages that when compressed by the at least oneprocessing device of the given enclosure would fill a stripe; reducing,by the storage controller, the estimated size based at least in part onthe size of the at least one other compressed data page found in theinformation; selecting, by the storage controller, one or more datapages of the additional data pages based at least in part on the reducedestimated size; and providing, by the storage controller, the selectedone or more data pages of the additional data pages to the at least oneprocessing device of the given enclosure.
 8. The method of claim 7wherein the information comprises an indication of a location of each ofthe one or more compressed data pages that was stored on the pluralityof drives and a size of each of the one or more compressed data pagesthat was stored on the plurality of drives.
 9. The method of claim 7wherein generating the compressed data pages based at least in part onthe received data pages comprises at least one of: causing a compressionoffload engine of the storage system to compress the received datapages; and compressing the received data pages by the at least oneprocessing device of the given enclosure.
 10. The method of claim 7wherein the method further comprises: receiving, by the at least oneprocessing device of the given enclosure, the selected one or more datapages of the additional data pages from the storage controller;generating, by the at least one processing device of the givenenclosure, additional compressed data pages based at least in part onthe selected one or more data pages; and storing, by the at least oneprocessing device of the given enclosure, the at least one othercompressed data page and one or more of the additional compressed datapages together in another stripe on the plurality of drives.
 11. Themethod of claim 7 wherein responsive to receiving, by the at least oneprocessing device of the given enclosure, a read command from thestorage controller that targets a compressed data page stored in theplurality of drives, the method further comprises: retrieving, by the atleast one processing device of the given enclosure, the targetedcompressed data page from the plurality of drives; decompressing, by theat least one processing device of the given enclosure, the targetedcompressed data page to generate a decompressed data page; providing, bythe at least one processing device of the given enclosure, thedecompressed data page to the storage controller.
 12. The method ofclaim 7 wherein responsive to receiving, by the at least one processingdevice of the given enclosure, a read command from the storagecontroller that targets a compressed data page stored in the pluralityof drives, the method further comprises: retrieving, by the at least oneprocessing device of the given enclosure, the targeted compressed datapage from the plurality of drives; and providing, by the at least oneprocessing device of the given enclosure, the targeted compressed datapage to the storage controller; wherein the method further comprises:receiving, by the storage controller, the targeted compressed data pagefrom the at least one processing device of the given enclosure; anddecompressing, by the storage controller, the targeted compressed datapage.
 13. The method of claim 7 wherein responsive an indication of adisk failure, the method further comprises: reading, by the at least oneprocessing device of the given enclosure, RAID parities from theplurality of drives; regenerating, by the at least one processing deviceof the given enclosure, the one or more compressed data pages based atleast in part on the RAID parities; decompressing, by the at least oneprocessing device of the given enclosure, the regenerated one or morecompressed data pages; and providing, by the at least one processingdevice of the given enclosure, the one or more decompressed data pagesto the storage controller.
 14. A computer program product comprising anon-transitory processor-readable storage medium having stored thereinprogram code of one or more software programs, the program code beingexecutable by a storage system, the storage system comprising aplurality of enclosures and a storage controller, each enclosurecomprising at least one processing device coupled to memory and aredundant array of independent disks (RAID) arrangement comprising aplurality of drives, the at least one processing device being separatefrom the RAID arrangement; the program code, when executed by thestorage controller of the storage system, causes the storage controllerof the storage system: to obtain data pages associated with at least oneinput-output request; and to provide the obtained data pages to the atleast one processing device of a given enclosure of the plurality ofenclosures; to issue a command to the at least one processing device ofthe given enclosure to perform at least one operation based at least inpart on the obtained data pages; the program code, when executed by theat least one processing device of the given enclosure, causes the atleast one processing device of the given enclosure: to receive theobtained data pages from the storage controller; responsive to receivingthe command from the storage controller, to generate compressed datapages based at least in part on the received data pages; to store one ormore of the compressed data pages on a stripe of the plurality of drivesaccording to the RAID arrangement; to determine that at least one othercompressed data page of the compressed data pages does not fit on thestripe of the plurality of drives and has not been stored on theplurality of drives; and to return information associated with thestorage of the one or more of the compressed data pages to the storagecontroller, the information comprising an indication that the at leastone other compressed data page does not fit on the stripe of theplurality of drives and has not been stored on the plurality of drivesand a size of the at least one other compressed data page, the storagecontroller being configured to utilize the information to access the oneor more of the compressed data pages stored on the plurality of drives;wherein the program code, when executed by the storage controller of thestorage system, further causes the storage controller of the storagesystem: to obtain additional data pages associated with anotherinput-output request; to determine an estimated size of data pages thatwhen compressed by the at least one processing device of the givenenclosure would fill a stripe; to reduce the estimated size based atleast in part on the size of the at least one other compressed data pagefound in the information; to select one or more data pages of theadditional data pages based at least in part on the reduced estimatedsize; and to provide the selected one or more data pages of theadditional data pages to the at least one processing device of the givenenclosure.
 15. The computer program product of claim 14 wherein theinformation comprises an indication of a location of each of the one ormore compressed data pages that was stored on the plurality of drivesand a size of each of the one or more compressed data pages that wasstored on the plurality of drives.
 16. The computer program product ofclaim 14 wherein generating the compressed data pages based at least inpart on the received data pages comprises at least one of: causing acompression offload engine of the storage system to compress thereceived data pages; and compressing the received data pages by the atleast one processing device of the given enclosure.
 17. The computerprogram product of claim 14 wherein the program code, when executed bythe at least one processing device of the given enclosure, furthercauses the at least one processing device of the given enclosure: toreceive the selected one or more data pages of the additional data pagesfrom the storage controller; to generate additional compressed datapages based at least in part on the selected one or more data pages; andto store the at least one other compressed data page and one or more ofthe additional compressed data pages together in another stripe on theplurality of drives.
 18. The computer program product of claim 14wherein responsive to receiving a read command from the storagecontroller that targets a compressed data page stored in the pluralityof drives, the program code, when executed by the at least oneprocessing device of the given enclosure, further causes the at leastone processing device of the given enclosure: to retrieve the targetedcompressed data page from the plurality of drives, and at least one of:decompress the targeted compressed data page to generate a decompresseddata page and provide the decompressed data page to the storagecontroller; and provide the targeted compressed data page to the storagecontroller.
 19. The apparatus of claim 1 wherein the at least oneprocessing device of the given enclosure is further configured: toreceive the selected one or more data pages of the additional data pagesfrom the storage controller; to generate additional compressed datapages based at least in part on the selected one or more data pages; andto store the at least one other compressed data page and one or more ofthe additional compressed data pages together in another stripe on theplurality of drives.
 20. The apparatus of claim 1 wherein determiningthe estimated size of data pages that when compressed by the at leastone processing device of the given enclosure would fill a stripecomprises determining the estimated size that corresponds to one of: asize that would fill a stripe after compression has been performed usinga first compression ratio that is an average of a set of availablecompression ratios; and a size that would fill a stripe aftercompression has been performed using a second compression ratio that isthe highest compression ratio of the set of available compressionratios.